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I. 



INTRODUCTION 



The purpose of this study is to examine the Naval 
Facilities Engineering Command's (NAVFAC) current use of 
construction contractor performance evaluations (Standard 
Form 1420) . That current use will then be analyzed using 
accepted theory about control systems, measurements, and 
performance appraisals. Based on the analysis some recom- 
mended improvements to NAVFAC ' s contractor performance 
evaluation system will be made. Appendix A defines some of 
the key words used throughout this study. 

A. RESEARCH QUESTIONS 

The primary research question is: how can NAVFAC make 

better use of the construction contractor performance evalu- 
ations? These secondary research questions will also be 
addressed : 

1. How does NAVFAC currently use the performance 
evaluations? 

2. What does the Federal Acquisition Regulation (FAR) say 
about the contractor performance evaluations? 

3. What amount of attention do NAVFAC ' s managers place on 
the evaluations? 

4 . Do the contractors have a formal system for evaluating 
their subcontractors? 

5. What do contractors think about the performance evalu- 
ation process? 

6. What alternative uses are there for the evaluations? 
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7. What percentage of contracts result in outstanding, 
satisfactory, and unsatisfactory evaluations. 

B. METHODOLOGY AND LITERATURE REVIEW 

Mail and telephone surveys were used to determine the 
current uses of and attitudes about contractor performance 
evaluations. Different mail surveys were sent to: 

1. NAVFAC ' s Assistant Resident Of f icers-in-Charge of Con- 
struction (AROICC's) who directly administer construc- 
tion contracts. 

2. Construction contractors who have previously done work 
for NAVFAC. 

3. Supervisory civil engineers who manage NAVFAC' s field 
offices . 

Field contract specialists were surveyed by telephone. 

Literature on control and measurement systems was 
reviewed and provided a body of theory which was used to 
analyze the research data. The FAR, Department of Defense 
FAR Supplemental (DFARS) , and NAVFAC' s Contracting Manual 
(P-68) were used to develop background on the requirement 
for issuing contractor performance evaluations. 

C. LIMITATIONS 

A search for literature about other contractor perform- 
ance evaluation systems produced no relevant information for 
this study. Some data located at NAVFAC 's Engineering Field 
Divisions (EFD) could not be provided due to the heavy 
seasonal workload demands. The survey population sizes were 
relatively small. The limitations on the use of data 
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generated from these small populations are discussed in 
Chapter IV. 



D. SUMMARY OF FINDINGS AND RECOMMENDATIONS 

This writer concluded that NAVFAC could significantly 
improve its use of the contractor performance evaluations. 
These recommendations for improvements are made: 

1. Issue a policy statement describing to contractors the 
purpose for the evaluation system. 

2. Issue contractors interim evaluations mid-way through 
the contract. 

3. Provide contractors copies of their final evaluations. 

4. Provide AROICC's training on how to complete the 
evaluations. 

5. Increase management emphasis on the evaluation 
process. 

6. Provide meaningful rewards to contractors who are 
rated outstanding. 

7.. Publicize the times evaluations are successfully used 
to deny contract awards to contractors with unsatis- 
factory performance records. 

8 . Improve the access to and use of contractor perform- 
ance evaluation information. 

9. Improve the Standard Form 1420. 



E. ORGANIZATION 

The rest of this study consists of: 

- Chapter II — Background information about NAVFAC' s 
contracting environment and the regulatory requirements 
for contractor performance evaluations. 

- Chapter III — Control, measurement, and performance 

appraisal theory, and three models of possible uses for 
the evaluations. 



10 



Chapter IV — The research data summary. 

Chapter V--An analysis of contractor performance evalua- 
tions as a control mechanism. 

Chapter VI — Conclusions and recommendations 

Supporting appendices. 
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II . BACKGROUND 



This chapter provides background information about: 



1 . 


NAVFAC' s construction mission and 

environment 


contracting 


2 . 


NAVFAC 's organization 




3. 


The Federal Acquisition Regulation (FAR) requirements 
for pre-award surveys and performance evaluations 


4 . 


Standard Form 1420, Performance 

Construction Contract 


Evaluation- 


5. 


The Small Business Administration (SBA) 
tractor responsibility determinations 


role in con- 



A . NAVFAC BACKGROUND 

The Navy Facilities Engineering Command maintains the 
U.S. Navy shore facilities throughout the world. NAVFAC 
employs more than 25,000 people to ensure its mission is 
properly performed. NAVFAC augments its forces by contract- 
ing out more than $2 billion worth of construction to 
civilian firms each year. NAVFAC 's goal is to ensure each 
contract is completed safely, on time, within cost and at 
satisfactory quality. 

To accomplish that goal NAVFAC is organized on the 
levels shown in the diagram below: 
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NAVFAC Headquarters provides overall guidance, makes 
policy decisions, and provides staff support to the 
Engineering Field Divisions (EFD) . Each EFD awards con- 
struction contracts, monitors contract progress, and assists 
field offices in their contract administrative efforts. The 
field offices make some contract awards and administer the 
construction contracts. Each field office is managed by a 
supervisory civil engineer. Contract specialists award the 
contracts and provide expertise in contractual matters. The 
contract administration is performed by civilian contract 
administrators and by Civil Engineer Corps Officers (Assis- 
tant Resident Off icers-in-Charge of Construction, AROICC's). 
Technical inspection of construction work is performed by 
construction specialists. 

Almost all NAVFAC construction contracts are awarded 
using a publicly advertised sealed bid process. Under this 
process NAVFAC advertises its prospective contracts using an 
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invitation for bids. Contractors compete for the contract 
award by submitting sealed firm-fixed-price bids. The 
contract is awarded to the lowest responsive responsible 
bidder. To be responsive a bidder must meet all the condi- 
tions of the bid invitation. To be determined responsible a 
bidder must be able to demonstrate the capacity and capabil- 
ity to accomplish the contract work. NAVFAC has little con- 
trol over who its low bidders are. It does have some 
control in determining whether a contractor is deemed 
responsible. Responsibility determinations are made during 
a pre-award survey. The next section describes the regula- 
tory requirements concerning pre-award surveys and contrac- 
tor performance evaluations. 

B. REGULATORY REQUIREMENTS 
1 . Pre-Award Surveys 

During a Pre-Award survey, a contract specialist 
will examine the apparent low bidders qualifications to 
determine whether or not that contractor is responsible . To 
be determined responsible a prospective contractor must meet 
several criteria, including "Have a satisfactory performance 
record." (FAR, 1984, par. 9.104(c)) Contracts shall be 
awarded to responsible prospective contractors only." (FAR, 
1984, par. 9.103(a)) The determination of a contractor's 
responsibility is important because: 

The award of a contract to a supplier based on lowest 
evaluated price alone can be false economy if there is 
subsequent default, late deliveries, or other 
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unsatisfactory performance resulting in additional 
contractual or administrative costs. While it is impor- 
tant that Government purchases be made at the lowest 
price, this does not require an award to a supplier solely 
because that supplier submits the lowest offer. A pros- 
pective contractor must affirmatively demonstrate its 
responsibility, including when necessary, the 
responsibility of its proposed subcontractors. (FAR, 
1984, par. 9.103(c)) 

2 . Contractor Performance Evaluations 



A major source of information about a contractor's 
past performance is their contractor performance evaluations 
from prior contracts. Evaluation of contractor performance 
is a special aspect of contracting for construction. The 
FAR says: 

(b) Preparation of Performance Evaluation Reports: 

(1) The contracting activity shall evaluate contractor 
performance and prepare a performance report for each con- 
struction contract of 

(i) $500,000 or more; 

(ii) $100,000 or more, if any element of per- 
formance was either unsatisfactory or 
outstanding; 

(iii) More than $10,000, if the contract was ter- 
minated for default; or 

(iv) $100,000 or more, if the contract was ter- 
minated for the convenience of the 
Government. 

(2) The report shall be prepared at the time of final 
acceptance of the work, at the time of contract termina- 
tion, or at other times, as appropriate, in accordance 
with agency procedures. Ordinarily, the evaluating offi- 
cial who prepares the report should be the person respon- 
sible for monitoring contract performance. 

(3) If the evaluating official concludes that a con- 
tractor's overall performance was unsatisfactory, the con- 
tractor shall be advised in writing that a report of 
unsatisfactory performance is being prepared and the basis 
for the report. If the contractor submits any written 
comments, the evaluating official shall include them in 
the report, resolve any alleged factual discrepancies, and 
make appropriate changes in the report. 

(4) The head of the contracting activity shall 
establish procedures which ensure that fully qualified 
personnel prepare and review performance reports. 
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(b) Review of performance reports. Each performance 
report shall be reviewed to ensure that it is accurate and 
fair. The reviewing official should have knowledge of the 
contractor's performance and should normally be at an 
organizational level above that of the evaluating 
official . 

(c) Distribution and use of performance reports. (1) 
Each performance report shall be distributed in accordance 
with agency procedures. One copy shall be included in the 
contract file. The contracting activity shall retain the 
report for at least six years after the date of the 
report . 

(2) Before making a determination of responsibility in 
accordance with Subpart 9.1, the contracting officer may 
consider performance reports in accordance with agency 
instructions. (FAR, 1984, par. 36.201) 

The Department of Defense FAR Supplement mandates 
that "Performance evaluations of construction contractors 
shall be used in making responsibility determinations." 
( DFARS , 1984, par. 36.201(c)) NAVFAC ' s P-68 offers this 
guidance about contractor performance evaluations: 

PERFORMANCE REPORT. Contractor performance reports 
are valuable contract records and should be prepared by 
qualified personnel in a careful and conscientious man- 
ner. These reports must be based on factual rather than 
subjective data. These reports frequently form the basis 
for the selection of contractors for the accomplishment of 
critical work. They are essential in findings of nonre- 
sponsibility for contractors that have done prior work for 
NAVFACENGCOM. OICC's are responsible for assuring evalua- 
tion reports are promptly and accurately completed and 
distributed. Unless the contractor correctly points out 
factual errors, performance evaluation reports are not to 
be revised merely to meet contractor objections; rather, 
the contractor's comments are to be attached to the 
evaluation report. (P-68, 1985, par. 6-206) 



C. STANDARD FORM 1420 

Standard Form 1420, PERFORMANCE EVALUATION-CONSTRUCTION 
CONTRACTS is used for reporting contractor performance. A 
blank SF 1420 is included as Appendix B. The form allows 
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space for general contract data including: contractor name, 
complexity and description of work, contract start and 
finish dates, contract award amount, contract change orders, 
and the extent of subcontracting. Part II allows the rater 
to evaluate six performance elements (Quality of Work, Time- 
liness of Performance, Effectiveness of Management, Compli- 
ance with Labor Standards, Compliance with Safety Standards, 
and Overall Performance) as either Outstanding, Satisfac- 
tory, or Unsatisfactory. Any marks of outstanding or 
unsatisfactory must be supported by a narrative on the back 
of the evaluation. The evaluations are normally filled out 
by the AROICC upon completion of the contract work. 



D. THE ROLE OF THE SMALL BUSINESS ADMINISTRATION 

Statistics provided by NAVFAC's Small Business Liaison 

indicate that small business concerns receive approximately 

80% of NAVFAC's contracts (dollar value) each year. 

Whenever a prospective contractor is a small business, the 
Small Business Act vests in the Small Business Administra- 
tion sole authority to make a determination of nonrespon- 
sibility. An affirmative responsibility determination by 
SBA is issued via the medium of a Certificate of Competen- 
cy (COC) . If SBA declines for any reason to issue a 
certificate of competency, the prospective contractor 
shall be rejected as not responsible. (P-68, 1985, par. 

4-407. 4(b) ) 

The Small Business Administration (SBA) has this role in 
determining a contractor's responsibility: 

(a) Upon determining and documenting that a 
responsive small business lacks certain element of respon- 
sibility (including, but not limited to, competency, capa- 
bility, capacity, credit, integrity, perseverance, and 
tenacity) , the contracting officer shall — 
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(1) Withhold contract award (see 19.602-3), and 

(2) Refer the matter to the cognizant SBA 
Regional Office in accordance with agency procedures . . . 

(c) The referral shall consist of — 

(1) A notice that a small business concern has 
been determined to be nonresponsible, specifying the ele- 
ments of responsibility the contracting officer found 
lacking, and 

(2) A copy of the solicitation, drawings and 
specifications, preaward survey findings, pertinent tech- 
nical and financial information, abstract of bids (if 
available) , and any other pertinent information that sup- 
ports the contracting officer's determination. (FAR, 
1984, par. 19.602-1) 

Once the referral has been made to the SBA, the 
following procedures are followed: 

(a) Within 15 business days (or a longer period 
agreed to by the SBA and the contracting agency) after 
receiving a notice that a small business concern lacks 
certain elements of responsibility, the SBA will take the 
following actions: 

(1) Inform the small business concern of the 
contracting officer's determination and offer it an oppor- 
tunity to apply to the SBA for a certificate of competency 
(COC) . 

(2) Upon timely receipt of the application and 
required documentation, send an SBA team to visit the con- 
cern to investigate it only for the specific elements of 
responsibility that the agency notice specified as 
lacking, and to make recommendations to the SBA Regional 
Administrator. 

(3) If the Regional Administrator plans to issue 
or recommend issuance of a COC, provide advance notice of 
the proposed action to the contracting officer together 
with a brief statement of the reasons for it. If the con- 
tracting officer disagrees with the proposal, resolve the 
disagreement as provided in 19.602-3. (FAR, 1985, par. 
19.602-2) 



E. SUMMARY 

This chapter described NAVFAC's mission, its organiza- 
tion, and its contracting procedures. The chapter also 
reviewed the regulatory requirements for pre-award surveys 
and contractor performance evaluations. Finally, the 
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chapter discussed the Standard Form 1420 and the Small 
Business Administration's role in determining a contractor's 
competency. The next chapter examines accepted theories 
about control and measurement systems and develops three 
models of potential uses for the contractor performance 
evaluations. The theory and models will be used later 
during the analysis of research data. 
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Ill . CONTROL AND MEASUREMENT THEORY 



The contractor performance evaluations provide a measure 
of the contractor's performance on a contract. The 
contractor performance evaluation process is intended to 
control the quality of construction contractors who receive 
NAVFAC contracts. This chapter examines accepted theories 
of measurement, control, and evaluation relevant to NAVFAC 's 
use of contractor performance evaluations. 

A . CONTROL THEORY 

Management control is defined as, "... The process 
through which managers assure that actual activities conform 
to planned activities. . . . The control process . . . en- 
ables managers to detect deviations from the plan in time to 
take corrective action before it is too late." (Stoner, 
1982, p. 592) This definition implies that even though one 
is "in control," actual performance can vary from planned. 
A manager exerts control when he takes action to eliminate 
variations from plan. 

All control systems have some common characteristics 

. . . any control system has at least these four 
components : 

1. An observation device that detects or observes and 
measures or describes the activities or other phenomena 
being controlled. The term for this component may be 
observer, detector , or sensor. 
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2. An assessing device that evaluates the performance of 
an activity or organization, usually relative to some 
standard or expectation of what should be, and identifies 
out-of-control activities and conditions. The term for 
this component is evaluator assessor, or selector. 

3 . A behavior modification device for altering or chang- 
ing performance if the need for doing so is indicated. 
This component may be called a director, modifier, or 
effector . 

4 . A means of transmitting information among the other 
devices. This component's term is communication network . 
(Anthony, Deardon, Beford, 1984, p. 8, underlines added 
for emphasis) 

This model portrays the functioning of a control system 
according to Anthony. 



CONTROL DEVICE 




The arrows in the model constitute the communication 
network. In this control system the detector, selector, and 
effector functions could all be performed by the same 
person, or each could be performed by different people. 
Within a control system: 

Individuals typically occupy one of three organiza- 
tional positions with respect to each information and 
control system they interact with: 

1. The measured and controlled position. This position 
involves being measured by the system. 
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2. The system maintenance position. Their primary task 
is seeing that the decision makers have the information 
they need. 

3. The decision maker position. This position involves 
receiving the information from the control system in order 
to make decisions. (Lawler and Rhode, 1976, p. 8) 

The control system affects the behaviors of people in 
each position in different ways. Usually, the behavior of 
greatest concern is that of the person being measured. 
Those who design control systems should consider the 
system's effects on behavior to ensure dysfunctional 
behaviors are not produced. 

All information and control systems have these charac- 
teristics in common: 

1. They all have some similar structural characteristics. 

a. They all collect, store, and transmit information 
in the form of abstract measures of reality. 

b. The collected abstract information is stored and 
transmitted in a specific form and with specific 
frequency. 

c. The summarized information is distributed to a 
specific, usually pre-determined, group of people. 

2. All information tries to accomplish the same thing — 
influence behavior. The crucial aspect of any control 
system is its influence on behavior. (Lawler and 
Rhode, 1976, pp. 5-6) 

Given that control systems have some basic similarities, 
one can ask, what qualities make one control system better 
than others? How are good control systems developed is 
another good question. A good place to start is deciding 
the purpose of the control system. 
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Some measure performance to provide management with the 
information they need to control employee's present 
behavior. Others provide top management with the inform- 
ation they need for long-range planning. And a third 
group provides ongoing feedback to employees about their 
job performance. (Lawler and Rhode, 1976, p. 38) 



B. CONTRACTOR PERFORMANCE EVALUATIONS AS A CONTROL SYSTEM 

Lawler and Rhode's previous quote described three 
possible uses for control systems. Let us now examine how 
NAVFAC's Contractor Performance Evaluation system could 
fulfill each of those three uses. 

1 . Use I: Award Decisions 



Providing top management with information for long- 
range planning is currently the actual purpose of NAVFAC's 
contractor performance evaluation system. The evaluations 
provide top management with information which is used to 
make decisions about whether contractors receive future con- 
tract awards. This model describes the control system: 



CONTROL SYSTEM 
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In this model the AROICC functions as the detector 



and evaluator. The OICC or EFD acts as the effector because 
the action of interest is the future contract award. 

2 . Use II: Controlling Ongoing Contractor Performance 

If NAVFAC issued interim performance evaluations to 
contractors, then their behavior on current contracts could 
be affected. This model depicts the evaluation system 
functioning as a control of a contractor's current behavior. 



CONTROL SYSTEM 




Notice in this model that the AROICC performs the 
detector, evaluator, and effector functions by himself. The 
AROICC is the effector because the action of interest is 
controlling the ongoing contractor performance. 

3 . Use III: Controlling ArOICC Performance 

Lawler and Rhodes third use, providing ongoing feed- 
back to employees about their job performance, could be 
satisfied under NAVFAC ' s current evaluation system. 
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Specifically, top management could control the AROICC's use 
of contractor performance evaluations by rating the AROICC's 
performance on: (1) the preparation of contractor evalua- 
tions, and (2) the usefulness of contractor performance 
evaluations as a source of information prior to administer- 
ing a contract. The AROICC's performance should improve 
because preparing a better contractor performance evaluation 
will require increased documentation and understanding of 
the contractor's performance. Reviewing the contractor's 
past performance evaluations prior to administering a new 
contract will alert the AROICC to that contractor's previous 
problems. This two-tiered model depicts that control possi- 
bility within the NAVFAC organizational structure: 



CONTROL SYSTEM 
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In this model, on the upper tier the OICC acts as 
the detector, evaluator, and effector. The OICC is the 
effector because the action of interest is controlling the 
AROICC's performance. 

The three models just presented include the four 
basic characteristics of control systems described by 
Anthony. The evaluation system also meets the criteria des- 
cribed earlier by Lawler and Rhode, because: 

- The evaluations collect, store, and transmit 
information. 

- The information is stored in a specific format (SF 1420) 
and is collected with specific frequency. 

- The summarized information is distributed to a specific 
group of people (OICC, EFD, Army Corps of Engineers) . 

- The evaluations purpose is to assist in determining the 
responsibility of contractors. If one assumes contrac- 
tors want to be determined responsible, then the evalua- 
tions can be assumed to have an influence on their 
behavior. 



C. CONSIDERATIONS IN DEVELOPING AND EVALUATING CONTROL 
SYSTEMS 

A control system provides the user with information in 
the form of a measure. The following items represent some 
characteristics which make information useful to its users: 

1. Usefulness for Decision Making — The characteristics of 
information that make it a desirable commodity can be 
viewed as a hierarchy of qualities, with usefulness 
for decision making of most importance (FASB, 1980, p. 
4037). There are many user-specific factors to con- 
sider when deciding what information is most useful 
for decision making. These factors include: " . . . 

the decisions to be made, the methods of decision 
making to be used, and the information already posses- 
sed or obtainable from other sources." (FASB, 1980, 
p. 4037) 
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2. Understandabil itv — "Information cannot be useful to 
decision makers who cannot understand it, even though 
it may otherwise be relevant to a decision and be 
reliable." (FASB, 1980, p. 4037) 

3. Relevance — "Does the'measure contain information per- 
tinent to the decision at hand?. . . . Looking for- 

ward, relevance implies predictive ability; 

Looking backward, relevance implies feedback value." 
(Moses, 1986, p. 9) "Relevant information is capable 
of making a difference in a decision by helping users 
to form predictions about the outcomes of past, 
present, and future events or to confirm or correct 
prior expectations. Information can make a difference 
to decisions by improving decision makers' capacities 
to predict or by providing feedback on earlier expec- 
tations. Without a knowledge of the past, the basis 
for a prediction will usually be lacking." (FASB, 

1980, p. 4038) 

4. Timeliness — "If information is not available when it 

is needed or becomes available so long after the 
reported events that it has no value for future 
action, it lacks relevance and is of little or no 
use." (FASB, 1980, p. 4038) "Timeliness does not 

cause relevance, but lack of timeliness can make an 
otherwise useful measure irrelevant." (Moses, 1986, 
P. 10) 

5. Reliability — "To what degree do repeated measurements 

of the same attribute vary?" (Euske, 1984, p. 86) 

"Reliability connotes stability, consistency, objec- 
tivity, dependability, and absence of bias. In short 
reliability is the relative absence of errors of 
measurement in the measurement instrument." (Moses, 
1986, p. 5) 

6. Requisite Variety — "Do the measures make sufficiently 

fine distinctions among objects? A given decision 

situation may require a particular kind of separation 
or differentiation among objects. If a measurement 
process is to provide useful information for the deci- 
sion, it should result in measures that capture the 
required distinctions." (Moses, 1986, p. 12) 

7. Completeness — "A crucial aspect of any control system 
is how completely and inclusively it measures the 
behaviors that need to be performed by a job holder. 

. . If measures are incomplete, a person will be 
motivated to perform only a portion of the behavior 
needed for organizational effectiveness." (Lawler and 
Rhode, 1976, p. 42) 
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8. Valence — "To what degree are the measurements tied to 

the organizational reward system? . . . Individuals 

act to satisfy personal needs. To the extent that 
some needs are satisfied by the organizational reward 
system, and to the extent that rewards are linked to 
specific measurements, behavior to influence the 
measurements will be motivated." (Moses, 1986, p. 16) 

9. Controllability — "Can the person measured influence 
the events or objects that are represented by the 
measurements? Without control, individuals cannot 
feel responsible for outcomes or feel satisfaction in 
outcomes, and consequently, will not be intrinsically 
motivated to perform. Without control individuals 
will see no link between his or her behavior and 
potential rewards, and consequently will not be 
extrinsically motivated." (Moses, 1986, pp. 16-17) 

10. Objectivity — "Some systems use objective measures 
while others use subjective ones. . . . Research 
shows that the more objective measures are, the more 
likely they are to motivate behavior. ... It is 
only logical that if individuals don't understand how 
the measure operates because it is highly subjective, 
they will see little connection between their behavior 
and any reward based on the measure. . . . Another 

reason objectivity of measures is important has to do 
with the climate of mistrust present in most organiza- 
tions. When employees do not believe they will be 
evaluated fairly, they do not believe that good per- 
formance on their part will lead to rewards, and as a 
result motivation is low." (Lawler and Rhode, 1976, 
pp. 42-47) 



D. OTHER CONSIDERATIONS FOR DESIGNING A CONTROL SYSTEM 

Once the purpose for a system is made clear, and all the 
user-specific needs are identified, the system designer can 
proceed. These other factors should be considered when 
developing a control system. Following each listed factor 
below is a description of that factor, and a discussion of 
what makes each factor a desirable feature of a control 
system. 
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1 . 



Nature of Standards — The research and theory on 
extrinsic motivation make one point quite clear: 
Employees are not motivated to reach goals that they 
don't feel they can achieve. . . . The research on 

goal setting (e.g., Locke, 1968) suggests that indi- 
viduals tend to achieve easy goals but not ‘exceed 
them. The result is mediocre performance. . . To set 
goals at the proper level of difficulty, individuals 
who have information on where the standards should be 
set must be involved. Further, the individuals whose 
performance is being measured must be aware of the 
information that was used to set the standards so they 
will realize that the standards are reasonable. 

2. Source of Discrimination — As with standard setting, it 

is crucial who acts as discriminator. . . . Whoever 

it is must have two attributes: the knowledge to make 

the comparisons and the trust of the person being 
measured. 

3. Pattern of Communication — Control systems vary widely 

in terms of who receives information about deviations 
from the standard. . . . For extrinsic motivation to 

be present, the person who has the power to give 
rewards must receive the information about the results 
of the discrimination. This is a precondition to 
rewards being allocated on the basis of performance 
which in turn is a precondition to the perception that 
rewards are based upon performance. However, for this 
perception to exist, ... it is desirable to have the 
person whose performance is being measured, and other 
employees who are doing similar work, receive the 
communication . 

4 . Speed and Frequency of Communication — Control systems 
differ in how quickly they report the results of their 
measurement process. ... As a general rule, commun- 
ications about performance measures are most effective 
in producing extrinsic motivation when they are fast 
and frequent. When the communications about perform- 
ance are delayed, it is impossible to closely tie 
rewards to the actual performance of the person and 
this has the effect of reducing the perceived rela- 
tionship between performance and rewards. It also 
creates doubts among employees about the validity and 
usefulness of the performance measurement data. 
(Lawler and Rhode, 1976, pp. 42-43) 

The theory on performance appraisals offers this advice: 

Today's employees want more from an organization than 

just their salary. They want to know how they are 
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performing in their jobs. . . . People do not mind being 

evaluated if the appraisal system is fair and comprehen- 
sive to them. (Cohen and Jaffee, 1982, p. 94) 

Wells says an effective and defensible performance 

i 

appraisal system should have these qualities: 

1. Understandable Standards — Every employee should under- 

stand the standards against which he or she will be 
evaluated. . . . They should have prior knowledge of 

what is expected. 

2. Knowledgeable Appraisers — Another problem with similar 
implications, may arise when immediate superiors do 
not possess sufficient technical knowledge or 
expertise in an area of specialization to reasonably 
judge the results produced by others. 

3 . Appraisers Should Be Well Trained — There are a number 

of reasons why it is critical that those conducting 
the appraisals should be well trained to guarantee 
consistency, . . . Consistency in administration is 

integrally linked to the defensibility of the system 
in the event of a legal challenge. 

4 . Communication of Policy and Purpose — The purposes and 
uses of appraisals should be clearly stated in the 
policy manual, as well as in the employee handbook. . 

. . Fully informing all employees of appraisal policy 
and purposes will minimize uncertainty and potential 
resistance. Employees understanding will increase the 
potential for positive results from appraisals. 
(Wells, 1982, pp. 777-781) 

The items Wells stresses are consistent with Lawler and 
Rhodes considerations for control systems. The theories and 
models presented in this chapter will be used in Chapter V 
during the analysis of NAVFAC's current use of performance 
evaluations. The next chapter presents the data generated 
during the research about NAVFAC's current use of contractor 
performance evaluations. 
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IV. RESEARCH DATA 



A. OVERVIEW 

This chapter describes the research methods used to 
examine NAVFAC's current use of Contractor Performance 
evaluations. The research data produced is also presented. 
The following sources of data were used: 

1. Mail survey of AROICC's. 

2. Mail survey of construction contractors. 

3. Telephone survey of field office contract specialists. 

4. Mail requests for information from EFD points of 
contact. 

5. Mail survey of field office Supervisory Civil 
Engineers . 

6. Small Business Administration sources. 

The mail and telephone surveys were prepared using 
Dillmans' Total Design Method (Dillman, 1982). The Total 
Design Method is a book which provides comprehensive 
instructions on how to prepare successful mail and telephone 
surveys. Among other things, the Total Design Method: (1) 

helped reduce evaluation bias, (2) gave tips for increasing 
the survey response rate, and (3) provided examples of 
properly phrased survey questions. The purpose for, back- 
ground about, and survey results for each data source are 
presented below. 
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B. AROICC SURVEY 



1 . Purpose 

A mail survey of AROICC 's was used to determine: 

- AROICC opinions about the effectiveness and usefulness 
of the contractor performance evaluations. 

- The amount of training AROICC' s have in completing an 
evaluation . 

- AROICC perceptions about management's emphasis on 
evaluations . 

- AROICC opinions on the use of interim evaluations. 

2 . Background 

Before the AROICC survey form was finalized, 8 
"pilot" surveys were sent to AROICC 's to determine if the 
responses would generate acceptable data. All 8 surveys 
were returned. A review of the responses resulted in the 
elimination of one question, and minor rewording of two 
others. The final survey (Appendix C) was sent to 60 
AROICC' s. The AROICC 's surveyed were chosen from NAVFAC's 
P-1, Navy Civil Engineer Corps Directory. Only LTJG's and 
above with more than 1 year experience as an AROICC were 
surveyed. 54 of the 60 (90%) surveys were returned. The 
data summary below includes the 8 pilot surveys, for a final 
response from 62 of 68 (91%) AROICC' s. 

3 . AROICC Survey Results 

The AROICC survey questions and summarized responses 
are presented below. 
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Question 1 . Approximately how many satisfactory, outstand- 
ing, and unsatisfactory contractor performance evaluations 
have you written during your career as a contract 
administrator? 

Totals: 1361’ (86%) — Satisfactory 

119 (7.6%) — Outstanding 

100 (6.4%) — Unsatisfactory 
1580 

The average respondent has filled out 25.4 evaluations 
each. 

Question 2 . Does your EFD have an instruction governing 
contractor performance evaluations? (All EFD's do have 
instructions . ) 

45 Of 62 (64%) — YES 

16 of 62 (26%) — NO 

1 of 62 — Did not answer 



Question 3 ♦ If answer to question 2 was yes, how much help 
does the instruction provide when you are filling out an 
evaluation? 



1 of 45 — A Big Help X 

22 of 45 — Some Help X 

16 of 45 — A Little Help X 
4 of 45 — No Help X 

2 of 45 — Did not answer 



3 points = 03 
2 points = 44 
1 point = 16 
0 points = _0 

65 points/43 
respondents = 
1.51 



The weighted average response of 1.51 falls mid-way between 
a little and some help. 



Question 4 . Have you ever received any training on how to 
complete the performance evaluation form? 



23 of 62 (38.1%) — YES 

39 of 62 (61.9%) — NO 

If YES, which of these statements most accurately 
describes the level of your training? 

4 — Formal Classroom training 

5 — Briefed during AROICC staff meeting 

11 — OJT by Contract Specialist or another AROICC 

12 — Given a copy of an old evaluation as a guide 
32* 



* — Some of the respondents listed more than one 
type of training 
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Question 5 . Do you ever review a contractor's record of 
past performance prior to the start of a new contract? 

23 of 62 (38.1%)— YES 

39 of 62 (61.9%) — NO 

*> 

If yes, approximately how often do you do so? 

12 do so more than 50% of the time 

11 do so less than 50% of the time 

Question 6 . Do you know where you could find a file of all 
of a particular contractor's past performance evaluations? 

25 of 62 (40%) — YES 

37 of 62 (60%) — NO 

Question 7 . On a scale of 1 to 10, how effective do you 
think the existing performance evaluation system is at 
improving the overall performance of NAVFAC contractors? (1 
not effective to 10 very effective) 

not effective very effective 

scale — 0 123456789 10 

# of — 1 11 11 20 7 3 2 3 0 0 1 

responses 

The average was 3.03. A trimmed average (omit the 
lowest and highest response) was 2.96. 73% of the responses 

were 3 or less. 

Question 8. Have you ever written an interim evaluation on 
a contractor mid-way through a project? 

15 Of 62 (24%) — YES 
47 of 62 (76%) — NO 

If yes, what effect does the interim evaluation have on a 
contractor's performance? 

10 of 15 said there was a noticeable improvement in the 
contractor's performance. 

Question 9 . Do you think interim evaluations would help you 
to positively influence a contractor's behavior? 

41 of 62 (67%) — YES 

18 of 62 (29%) — NO 

3 of 62 ( 4%) — Did not answer 
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Why? The YESES typically said: 



a . 


Contractor will adjust behavior 
NAVY 


to please 


the 


b . 


Early feedback will prove 
contractor 


useful 


to 


the 


c . 


Will provide an evaluation 
AROICC is transferred 


on 


file 


if 


the 


d. 


Contractors are concerned 

reputations 




about 


their 



The NO's typically said: 

a. Contractors are only motivated by money 

Question 10 . Which of these statements most accurately 
describes how much emphasis your office management places on 
the performance evaluations? 



18 


of 


62 


19 


of 


62 


14 


of 


62 


5 


of 


62 


5 


of 


62 


1 


of 


62 



(29%) — Evaluations are stressed frequently 
(31%) — Evaluations are stressed occasionally 
(22%) — Evaluations are stressed if contractor 
performance unsatisfactory 
( 8%) — Evaluations are stressed when overdue to 
the EFD 

( 8%) — Evaluations are never stressed 
( 2%) — Did not answer 



Question 11 . What comments do you have about the way we 
evaluate our contractors? Please include any suggestions 
for improving the evaluation process. 



This is a summary of the comments received. 



Frequency 

13 



11 



6 



5 



3 



Comment 



The evaluations are not effective because 
NAVFAC issues subsequent awards to contractors 
who've gotten unsatisfactory evaluations 

The evaluation forms need more performance and 
rating categories to allow for a more detailed 
evaluation of the contractor's performance 

The EFD's need to issue periodic summaries of 
contractor performance for use by field 
offices 

Unsatisfactory evaluations are avoided because 
they take too much time, effort, and documen- 
tation to support 

The evaluations are too subjective 
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3 A policy should be established to mandate 

denial of awards after a certain number of 
unsatisfactory evals 

3 We need meaningful rewards for outstanding 

contractors 

2 The Small Business Administration rarely rules 

against a contractor in a Certificate of 
Competency case 

2 Interim evaluations should be sent to 

contractors 

2 AROICC's should be better trained in filling 

out evaluations 

2 Time constraints affect the quality of 

evaluations 

1 The narrative should be written in bullet 

statements 

1 Need some criteria for translating separate 

ratings on each item into an overall rating 

1 A contractor's performance is affected by the 

quality of the project design 

1 Copies of the evaluations should be sent to 

the contractors' bonding companies 



CONTRACTOR SURVEY 
1 . Purpose 

The contractor survey was used to determine: 

Contractor opinions about the quality of NAVFAC's 
performance evaluations. 

Contractor opinions about the performance evaluation 
process and how it might be improved. 

Contractor systems for evaluating subcontractors 

Contractor opinions about receiving interim evaluations. 
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2 . Background 



Time constraints prevented the use of a "pilot" 
survey. Contractor mailing lists with contractor names, 
addresses, and points of contact were requested from each 
EFD. Contractors on the lists were to have regularly per- 
formed contracts within the EFD. 4 of 6 EFD's provided the 
contractor lists, but only 2 of 4 included names of 
contractor points of contact. Using those lists, surveys 
were sent to 80 contractors. 32 of the 80 (40%) returned 
the surveys, but 13 did not complete them because they had 
never seen nor received a performance evaluation from 
NAVFAC. Of the 19 who did complete the surveys, 10 contrac- 
tors had seen or received an evaluation, and 9 had not. A 
copy of the contractor survey is included as Appendix D. 

3 . Contractor Survey Results 

The contractor survey questions and summarized 
responses are presented below. 

Question 1 . On a scale of 1 (no value) to 10 (high value) , 
how much value do you assign to the performance evaluation 
rating you receive on a project? 

no value high value 
scale — 1 23456789 10 
# Of — 4 00030143 4 
responses 

The overall average was 6.6. The average for those who have 
seen an evaluation before was 7.9. 63% of the responses 
were 7 or above. 
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Question 2 . How useful to you think interim evaluations 
would be in providing you feedback about your performance on 
a contract? 



9 of 19 (47%) — Very Useful 

4 of 19 (21%) — Fairly Useful 

3 of 19 (16%) --Useful 
2 of 19 (11%) — Of Little Use 
1 of 19 ( 5%) — Did Not Answer 

Of the 10 who've seen an evaluation before, 9 of 10 said 
fairly or very useful. 

Question 3 . Would you like to see NAVFAC issue interim 
performance evaluations on its contracts? 

13 of 19 (68%) — YES 
6 of 19 (32%) — NO 

Why? The yes's said: 

a. It would eliminate any surprises in the final 
evaluation 

b. It would improve AROICC/contractor communications 

c. Contractors need feedback to know if work satisfies 
the government 

d. Contractors can correct mistakes before it is too 
late 

The no's said: 

a. AROICC's are not qualified to make the evaluations 

b. Contractors already know how good their performance 
is. 



Question 4 . Do you have a system for evaluating your 
subcontractors? 

None of the respondents have a formal evaluation system. 
All make basic subjective judgments about the subcontrac- 
tor performance. Those who don't perform well are not 
used on future jobs. 

Question 5 . Would you find it useful to receive a copy of 
your performance evaluation for each contract you perform? 

18 of 19 (95%) — YES 
1 of 19 ( 5%)— NO 

Comments : These general comments were made by contractors 

who were surveyed: 
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Frequency Comment 

2 Evaluations are improperly used by AROICC's as 

a bargaining tool at contract close out 

2 Evaluations are too subjective 

2 Evaluations are poorly documented 

1 Contractors should also evaluate the AROICC's 

1 An appeal process is needed to mediate 

disputed evaluations 

1 Evaluations don't matter because poor 

contractors still receive contract awards. 



D. CONTRACT SPECIALIST SURVEY 

1 . Purpose 

A telephone survey of contract specialists in 20 of 
NAVFAC's field contract offices was used to determine: 

- How performance evaluations were used in support of the 
pre-award survey process. 

- If the evaluations provided contract specialists enough 
information to determine contractor responsibility. 

- Contract specialists attitudes about the level of 
management emphasis on performance evaluations. 

- Contract specialists' use of the EFD performance 
evaluation files. 

2 . Background 

Survey questions were asked uniformly to all 20 
respondents, and answers were consistently recorded. 5 of 
the 20 respondents were not capable of answering some of the 
questions because of their offices' unique organizational 
structure. This information is used with the caveats that: 
(1) the survey population is statistically quite small, and 
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(2) the size and volume of contracts administered in each 
surveyed office was not determined. A copy of the survey is 
included as Appendix E. 

3 . Contract Specialist Survey Results 



The contract specialist survey questions and summar- 
ized responses are presented below. 

Question 1 . What percent of the time do you use a 
contractor's past performance evaluations to assist you 
during the Pre-Award Survey process? 

1 of 20 ( 5%) — Use the evaluations 100% of the time 
8 of 20 (40%) — Use the evaluations if the contractor 

has done work in their office before 
11 of 20 (55%) — Use the evaluations less than 10% of the 

time. They prefer to rely on phone 
calls to contractor provided references 
for information about a contractor's 
past performance. 

Question 2 . What percent of the time do you discuss the 
pre-award survey results with the AROICC assigned to the 
pro j ect? 



1 of 20 
4 of 20 
11 of 20 



4 of 20 



( 5%) — The AROICC does the pre-award survey 
(20%) — Notify the AROICC 90% of the time 
(55%) — Notify the AROICC if the contractor's 
past performance is less than 
satisfactory 

(20%) — Do not notify the AROICC at all 



Question 3 . Other than your own files, do you use any other 
sources of contractor performance evaluations? 



14 of 20 (70%) — Use no other source 
5 of 20 (25%) — Use their EFD's file less than 5% of the 

time 

1 of 20 ( 5%) — Does not even keep its own file of 

evaluations 
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Question 4 . Do the performance evaluations give you enough 
information to do the pre-award surveys properly? 

8 of 20 (40%) — Said the information was adequate 
7 of 20 (35%) — Said the information on the evaluations 

was too general, and that more specific 
comments about contractor performance 
should be made 

5 of 20 (25%) — Provided an inadequate response 

Question 5 . How much management emphasis is placed on 
performance evaluations within your office? 

4 of 20 (20%) — Strong management emphasis 

7 of 20 (35%) — A fair amount of management emphasis 

5 of 20 (25%) — A little management emphasis 

4 of 20 (20%) — Provided an inadequate response 

Question 6 . In the past year how many contractors did your 
office withhold awards from because of an unsatisfactory 
record of past performance? 

12 of 20 (60%) — Zero 
5 of 20 (25%) — One 
2 of 20 (10%) — Four 
1 of 20 ( 5%) — Six 



E. EFD INFORMATION 

The following information was gathered from the EFD 
points of contact in response to mail and telephone 
requests. 1 of the 6 EFD's chose not to provide any 
information . 

1. All 5 EFD's have instructions governing the use and 
preparation of contractor performance evaluations. 

2. 2 of 5 EFD instructions encourage the use of interim 

evaluations if the contractor's performance is 

unsatisfactory. 

3. All EFD's keep past performance evaluations separated 
by contractor. 

4. Pacific Division augments the SF1420 with its own 
"Supplemental Information Sheet" (Appendix F) . This 
sheet allows AROICC's to rate contractors on more 
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specific items. The sheet also has two extra ratings, 
marginal and exceptional. 

5. Pacific Division keeps an evaluation record form 
(Appendix G) on each contractor. That form summarizes 
a contractor's performance evaluations from many 
contracts . 

6. Northern Division is trying to implement a computer 
data base which will create a separate file of evalua- 
tion data for each contractor. The data base will be 
accessible to all of Northern Divisions field offices. 



F. SUPERVISORY CIVIL ENGINEER SURVEY 

1 . Purpose and Background 

A mail survey of 10 supervisory civil engineers was 
conducted to assess their attitudes about contractor per- 
formance evaluations. 7 responses were received (70%) . A 
copy of the survey is included as Appendix H. Several of 
the survey questions were not well prepared and their 
responses were unsuitable for use. This information is used 
with the caveat that the sample population is statistically 
very small. 

2 . Supervisory Civil Engineer Survey Results 

The supervisory civil engineer survey questions and 

summarized responses are presented below. Those questions 

which were inadequately prepared are not included here. 

Question 3 . Do you provide your AROICC's any training on 
how to complete the performance evaluation form? 

3 of 7 said YES 

3 of 7 said NO 

1 of 7 does all evaluations himself, therefore no 
training needed 



42 



Question 4 . Does your office keep separate performance 
evaluation files on each contractor? 

5 of 7 said NO 
2 of 7 said YES 

Question 6 . Does your office write interim evaluations on 
contractors midway through a report? 

5 of 7 said NO 
2 of 7 said YES 



Question 7 . Do you think NAVFAC should issue interim evalu- 
ations on all of its construction contracts? 

6 of 7 said NO — They fear the field office staff already 

has more work than it can comfortably 
handle 

1 of 7 said YES 

Question 9 . If you had access using the office computer to 
a file listing all of a contractor's past performance 
evaluations, would you use it to get a feel for how a pros- 
pective contractor of yours usually performs? 

6 of 7 said YES 
1 of 7 said NO 

Question 10 . Do you think we should give our contractors a 
copy of their performance evaluations at the end of a 
contract? 



5 of 7 said YES 
2 of 7 said NO 

Question 11 asked for comments on how NAVFAC could improve 
its evaluation system. These responses were received: 



Frequency 

2 

2 



1 



Comment 

The evaluations should include more specific 
items, and have more rating categories 

Evaluations have no effect on contractors 
because the poor performers continue to get 
new contract awards 

The excessive amount of documentation needed 
to support an unsatisfactory rating is 
burdensome 
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1 The form should require an explanation of a 

contractor's satisfactory performance also 

1 Evaluations are given superficial treatment by 

AROICC's because it takes too much time to 
prepare a good evaluation 

G. SMALL BUSINESS ADMINISTRATION DATA 

1 . Purpose and Background 

When small businesses are involved the Small 
Business Administration has the authority to override a 
NAVFAC decision of contractor non-responsibility. That 
override occurs when a Certificate of Competency is issued. 
2 of the AROICC's responding to the survey said the SBA 
rarely rules against contractors on COC decisions. The data 
presented below comes from the General Accounting Office 
report #RCED-86-120BR dated April 1986 and titled "Small 
Business Administration-Status, Operations, and Views of the 
Certificate of Competency Program." The report highlights 
actual uses of the contractor performance evaluations within 
the SBA's COC system. 

2 . SBA Data 

All data presented below comes from the above cited 
GAO report. The data represents all types of contracts 
reviewed by the SBA including some construction contracts. 
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a. This table provides statistics on the COC activity for 
1981-1985. 



Fiscal 

Year 


Refer- 
rals 
to SBA 


Contractor 

Applica- 

tions 


COC 

approv- 

als 


Applica- 

tion 

denials 


Direct 
Awards 
without 
a COC 


1981 


2652 


880 


489 


322 


12 


1982 


2837 


998 


495 


409 


13 


1983 


2949 


1071 


556 


393 


20 


1984 


3072 


1099 


547 


378 


29 


1985 


4223 


1652 


884 


540 


54 


Totals 


15733 


5700 


2971 


2042 


128 



The numbers of approvals, direct awards, and denials do not 
equal the number of applications because of such things as 
applications withdrawn, referrals withdrawn, and procure- 
ments withdrawn. 

Based on the total figures for the five years, there were 
10,033 contractors who did not apply for COC's, and another 
2042 who were denied COC's. Those 12,075 contractors repre- 
sent 7 6% of the total referrals to the SBA. Thus 7 6% of 
contractors referred to the SBA do not receive the contract 
award'. 

b. The GAO report included a survey of 279 cases. The 
survey looked at the processing time for those cases. The 
FAR allows the SBA 15 days to process a case, or a longer 
time if agreed to by the referring contracting agency. The 
results of the survey showed: 



Number of 
Workdavs 




Cases 

Number 


Percent 


15 or less 




128 


44.6 


16-20 




84 


29.3 


21-25 




32 


11.1 


26-35 




17 


5.9 


36 or more 




18 


6.3 




Total 


279 


100 



Though only 44.6% of the cases were completed within the 
allowed 15 days, 85% were completed within 25 days. 

c. The SBA's contractor review procedures are generally 
consistent with DOD's and GSA's review procedures. 
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d. No statutory or regulatory criteria exist to guide SBA 
in evaluating prior performance when making COC decisions. 
In addition, SBA has not established specific written 
criteria to make such determinations. The SBA evaluators 
must rely on professional judgment much like the AROICC's 
do . 

e. Of 287 cases sampled, 109 were referred to the SBA 
because of the contractors poor prior performance. Of those 
109, SBA agreed with the referring agency in 51 cases, or 
about 47%, and denied the COC. 



H . SUMMARY 

This chapter introduced data from several sources about 
NAVFAC's current use of the contractor performance evalua- 
tions. The next chapter provides an analysis of that data 
from the viewpoint of accepted control and measurement 
theory from Chapter III. 
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V. ANALYSIS OF CONTRACTOR EVALUATIONS AS A CONTROL MECHANISM 



Chapter III presented some accepted control and measure- 
ment theory as well as three possible uses for the contrac- 
tor performance evaluations. Chapter IV presented data 
about NAVFAC's current uses of the contractor performance 
evaluations. This chapter provides an analysis of those 
current uses from the viewpoint of the accepted theories. 
The analysis focuses on the three possible uses for the 
evaluations from Chapter III. 

A. USE I — AWARD DECISIONS 

The FAR, DFARS, and NAVFAC P-68 each cite only one 
purpose for the contractor performance evaluations; to 
provide information about contractor past performance during 
the pre-award survey. This control model reproduced from 
Chapter III depicts the control system. 



CONTROL SYSTEM 




award 

decisions 
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2 . Analysis 



When analyzed from the viewpoint of accepted control 
theory, NAVFAC ' s current use of the contractor performance 
evaluation system for making contract award decisions is 
weak in the following areas. 

a. Standards 

Lawler and Rhode, and Wells said control and 
appraisal systems should have well-defined standards. The 
standards should be understood, and should not be too easy 
or too difficult to attain. The FAR, DOD FAR Supplement, 
and NAVFAC P-68 all are silent about standards for contrac- 
tor performance. No guidance exists which defines what con- 
stitutes satisfactory, unsatisfactory, or outstanding 
performance. Instead, each AROICC is left to exercise his 
own professional judgment when evaluating a contractor's 
performance. 

b. Usefulness For Decision Making 

This characteristic of information is most 
important. Despite its importance, 11 of 20 contract 
specialists use the evaluations less than 10% of the time 
when doing pre-award surveys, and 14 of 20 do not use other 
available sources of evaluations. 5 of 7 supervisory civil 
engineers said their offices don't keep separate files of 
performance evaluations. These facts indicate NAVFAC could 
improve its use of the contractor performance evaluations in 
the contract award decision making process. 
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The following association of responses by the 
contract specialists provides some insight into the useful- 
ness of the performance evaluations. The responses to 
question 1 (What '% of the time do you use the evaluations 
during the pre-award survey process?) were matched against 
the responses to question 6 (In the past year how many con- 
tractors did your office withhold awards from because of an 
unsatisfactory record of past performance?) These results 
show that offices which used the evaluations more often were 
able to support more contract award denials: 

Question 6 — Number of contractors denied awards last year 

0 12 4 6 

Question 1 — What % of: Less than 10% — 83000 

the time do you use 

evaluations during More than 10% — 42021 

pre-award surveys? 

The offices who use the performance evaluations 
more often during the pre-award survey process reported a 
higher incidence of denying contractors awards, 
c. Communication 

Wells says employees should be told the purpose 
and uses of performance appraisals to minimize uncertainty 
and resistance. The results of the contractor survey 
clearly indicate the contractors are unaware that the evalu- 
ation system even exists (22 of 32 had never received an 
evaluation) . Lawler and Rhode say those being evaluated 
should receive the evaluations. NAVFAC currently notifies 
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contractors of their performance evaluation only if the per- 
formance is unsatisfactory (6.4% of the contracts). 

d. Training 

Wells says that appraisers should be well- 
trained. Only 38% of AROICC's surveyed said they'd been 
trained in completing evaluations (most of that training was 
informal) . Without common training NAVFAC is left with 
AROICC's who hold a wide variety of ideas about: 

- The intent of each rating category 

- The amount of detail needed to support an evaluation 

- The kinds of documentation needed to support an 
evaluation 

- The standards of performance for various ratings 

This wide variety of ideas reduces the relia- 
bility of the evaluation system. Recall that reliability 
connotes stability, consistency, objectivity, dependability, 
and absence of bias. Decreased reliability leads to 
decreased credibility when the SBA reviews the performance 
evaluations . 

e. Completeness 

Incomplete measures may motivate people to per- 
form only a portion of the behavior needed for organization- 
al effectiveness. Standard Form 1420 allows contractors to 
be rated on: 

- Quality of Work 

- Timely Performance 

- Effectiveness of Management 
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- Compliance With Labor Standards 

- Compliance With Safety Standards 

While these broad categories cover the range of 
important elements in contractor performance, they lack 
specificity, and therefore, fail to provide an adequate 
assessment of contractor performance. Contract specialists 
(7 of 15), AROICC's (11 of 62), and supervisory civil 
engineers (2 of 7) said the evaluations would be more 
valuable if more specific evaluation items were used. 
Though these numbers don't represent a majority of the 
survey populations, they are significant because the 
responses were made to an open-ended question. Within the 
context of the model, the OICC as decision maker would 
receive more information if more specific items were 
evaluated. 

f. Requisite Variety 

The OICC as decision maker must look at a con- 
tractor's record of past performance and decide whether or 
not to give him another award. As currently used, a 

marginal contractor, a satisfactory contractor, and a very 
good contractor all receive a satisfactory rating. The 
system does not allow a distinction between those levels of 
performance. Contract specialists (7 of 15) and AROICC's 
(11 of 62) said there should be additional ratings (i.e., 
marginal and highly satisfactory) . These were also 
responses to an open-ended question. These extra categories 
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would provide OICC's more precise information about a con- 
tractor's past performance, 
g. Valence 

To what degree are the measurements tied to the 
organizational reward system? Of 20 offices surveyed, 12 
withheld no contract awards due to unsatisfactory contractor 
past performance. Only 3 of 20 withheld more than 1 
contract. 22 of 32 contractors responding had never seen a 
performance evaluation before. 13 of 62 AROICC's commented 
(in response to an open-ended question) that the evaluations 
were not useful because unsatisfactory contractors continue 
to receive awards. 3 of 62 said NAVFAC needs a meaningful 
reward system to recognize its outstanding contractors. 
Clearly there is an absence of valence within the evaluation 
system. 

B. USE II — CONTROLLING ONGOING CONTRACTOR PERFORMANCE 
1 . Review 

NAVFAC' s use of contractor performance evaluations 
to control ongoing contractor performance is analyzed below. 
This model reproduced from Chapter III depicts the control 
system: 
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CONTROL SYSTEM 




evaluates 

performance 



2 . Analysis 

The FAR, DFARS , and NAVFAC P-68 do not acknowledge 
this potential use for contractor performance evaluations. 
2 of the 5 EFD's surveyed (Pacific and Atlantic) do offer 
guidance for issuing interim evaluations as feedback to con- 
tractors. When analyzed from the viewpoint of accepted con- 
trol theory, NAVFAC' s current use of the contractor 
performance evaluation system to control ongoing contractor 
performance is weak in these areas, 
a. Relevance 

Looking backward, relevance implies feedback 
value. NAVFAC notifies a contractor of his performance 
evaluation only if his performance was unsatisfactory (6.4% 
of the time) . That notification is made after the contract 
is already completed. To the contractor as decision maker 
(deciding how to behave) , that information is not relevant 
(because it is not timely) . The behavior in question is 



53 



completed and cannot be changed. Within the context of this 
model the performance evaluations are irrelevant. 

Interim evaluations would provide contractors 
relevant information. 10 of 15 AROICC's who 1 ve used interim 
evaluations before observed a noticeable improvement in 
contractor performance. 13 of 19 contractors said they'd 
like to receive interim evaluations because they provide 
feedback on their performance. 

b. Usefulness For Decision Making 

18 of 19 contractors said they'd like to receive 
copies of their final performance evaluation. The contrac- 
tors also placed a fairly high value on their evaluations 
they've received in the past (average 7.9 on a scale of 1 to 
10) . Although the final evaluations would not provide 
timely information about their performance on the ongoing 
contract, it would give the contractors feedback necessary 
to adjust their performances on future contracts. 

c. Understandability 

Information cannot be useful to decision makers 
who cannot understand it. The contractors as decision 
makers must be able to understand the evaluations they 
receive. The lack of standards, training, and a policy 
statement to contractors about the uses of evaluations 
decreases their understandability. 
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d. Requisite Variety 

This factor is a problem within the context of 
this model also. "Individuals tend to achieve easy goals 
but not exceed them." (Lawler and Rhode, 1976, p. 42) 
Since NAVFAC rates both marginal and very good contractors 
as satisfactory, marginal performance has become the "easy 
goal" of contractors. With more rating categories, the 
marginal performers can be identified, and all contractors 
can strive to become exceptional or outstanding. 

e. Completeness 

Recall that incomplete measures motivate people 
to perform only a portion of the behavior needed for organi- 
zational effectiveness. More specific rating items will 
provide contractors with more specific, and thus more 
complete feedback. The contractors will then be able to 
focus their efforts on all the behaviors needed to complete 
the contracts according to NAVFAC' s standards. 

f. Controllability 

The contractor has control of his own behavior. 
However, if he does not know what behavior is acceptable 
because he lacks feedback, the evaluation he receives is not 
completely within his control . With interim evaluations the 
contractor would have some very definite feedback mid-way 
through his performance, and could modify his behavior 
accordingly to improve his final evaluation. 
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C. USE III — CONTROLLING AROICC PERFORMANCE 
1 . Review 

Recall in this model that the AROICC uses informa- 
tion from contractors' past performance evaluations to pre- 
dict future contractor behavior. That future behavior can 
then be anticipated and better managed by the AROICC. 
Within this model the OICC places management emphasis on the 
completion and use of contractor performance evaluations by 
including them as a criteria for the AROICC 's personal per- 
formance rating. This model reproduced from Chapter III 
depicts the control system. 



CONTROL SYSTEM 




Performance 

Rating 
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2 . Analysis 



The data shows that only 38% of the AROICC's have 
referred to a contractor's past performance evaluations 
before the start of a new contract. 4 0% of AROICC's said 
evaluations are stressed by management only when they are 
overdue or unsatisfactory. 45% of contract specialists said 
management placed less than a fair amount of stress on 
evaluations. When analyzed from the viewpoint of accepted 
control theory, NAVFAC's current use of the contractor 
performance evaluations to control AROICC performance is 
weak in the following areas. 

a. Usefulness For Decision Making 

The evaluations can be used for many purposes 
within the context of this model. Recall that 60% of the 
AROICC's surveyed did not know where to find copies of the 
evaluations. 61% of the AROICC's never refer to contrac- 
tors' past evaluations prior to administering a contract. 
That knowledge about a contractor's history of performance 
would be useful when developing a strategy for administer- 
ing and inspecting a contract. Additional attention to weak 
areas of performance may avoid repeating past problems. 
These comments by Schmode, the supervisory civil engineer at 
New London, CT provide one view on this subject (see 
Appendix I for a copy of Schmode 's complete memorandum): 

The specific report required by reference (a) is fine for 
the purpose intended but does not provide us with the 
information that would prove useful in fulfilling our 
administrative responsibilities for follow-on contracts 
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with the same contractor. The five performance elements 
identified in the reference (a) required report (SF 1420) 
are much too general and the wide dispersion between 
"satisfactory" and "unsatisfactory" leave much to be 
desired. Basically, we need to know who our marginal 
contractors are, who our satisfactory contractors are that 
have some marginal or unsatisfactory traits. If we know 
in advance of the specific areas in which our contractors 
have proven themselves to be less than satisfactory, we 
can give our attention to those specific areas including 
emphasis during the Preconstruction Conferences. In 
short, we need a system that can be accurate, objective, 
consistent, require minimum effort, and can be cumulative. 
(Schmode, 1985, p. 1) 

The remainder of Schmode' s memorandum describes such a 
system. 

b. Reliability 

26% of the AROICC's surveyed did not know their 
EFD's had an instruction covering contractor performance 
evaluations. Only 38% of AROICC's had ever received 
training on how to complete an evaluation. These facts 
indicate there may be a lack of consistency, stability, and 
dependability in the evaluations prepared by those AROICC's. 
If OICC's use the contractor evaluations as a rating criter- 
ia of the AROICC, then the AROICC's will be motivated to 
prepare higher quality evaluations. These higher quality, 
more reliable evaluations will consequently improve the use- 
fulness of the evaluations for the other two purposes previ- 
ously discussed. 

c. Requisite Variety and Completeness 

Increased management emphasis on evaluation 
preparation will cause AROICC's to evaluate contractors more 
thoroughly on all elements of their performance. The 
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resultant evaluations will improve their utility for their 
other two uses. 

d. Valence 

When asked to rate the evaluations ' s effective- 
ness for improving contractor performance a majority of the 
AROICC's said the evaluations were not effective (trimmed 
average 2.96 on a scale of 1 (not effective) to 10 (very 
effective)). 40% of the AROICC's said management emphasizes 
the evaluations less than occasionally. 13 of 62 AROICC's 
said the evaluations are not effective because contractors 
rated unsatisfactory continue to get awards. There is a 
general impression that the evaluations do not make any 
difference. This low valence level adversely affects the 
quality of evaluations prepared and the use of evaluations 
by the AROICC's. 

D. SUMMARY 

This chapter analyzed each of the three possible uses 
for contractor performance evaluations from the viewpoint of 
accepted control and measurement theory. The next chapter 
uses that analysis as a basis for some conclusions and 
recommendations about NAVFAC's current use of construction 
contractor performance evaluations. 
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VI . CONCLUSIONS AND RECOMMENDATIONS 



The previous five chapters included the background, 
theory, data, and analysis about NAVFAC ' s current use of 
contractor performance evaluations. This chapter draws some 
conclusions about that use and makes recommendations on how 
NAVFAC can improve its use of the evaluations. 



A. CONCLUSIONS 

After reviewing the information in previous chapters, 
the following conclusions are made about NAVFAC ' s current 
use of contractor performance evaluations: 

1. NAVFAC lacks standards of performance which clearly 
describe the difference between unsatisfactory, satis- 
factory, and outstanding performance. 

2. The constructions contractors are not generally aware 
of the performance evaluation system. 

3. The evaluations are not used to provide contractors 
feedback about their performances. 

4. The evaluations are not timely, because they are 
written after the performance is completed. 

5. AROICC's are not well trained in completing the evalu- 
ations. The lack of training adversely affects the 
reliability of the evaluations. 

6. Management emphasis on the evaluation process could be 
improved. 

7. There is no well publicized incentive for contractors 
to provide outstanding performance. 

8. The evaluations are under-utilized for each of the 
three possible uses. 
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9. The Standard Form 1420 is not specific enough in iden- 
tifying the elements of contractor performance to be 
evaluated. 

10. The data base of information from the evaluations is 
inadequate. 



B. SHORT-TERM RECOMMENDATIONS 

These recommendations for improving NAVFAC ' s contractor 
performance evaluation system could be implemented quickly. 

1. Policy Statement — NAVFAC should issue a policy state- 
ment describing: 

a. The uses of the contractor performance 
evaluations . 

b. The standards of performance required for satis- 
factory, outstanding, and unsatisfactory perform- 
ance ratings. 

c. An explanation of the evaluation elements. 

This policy statement would increase contractor 
awareness of the evaluation system and improve the consis- 
tency, completeness, reliability, and usefulness of the 
evaluations. A draft policy statement is included in 
Appendix J. 

2 . Interim Evaluations — To provide contractors timely 
feedback about their contract performance, NAVFAC 
should require field offices to issue interim evalua- 
tions. The Standard Form 1420 can be used with a 
cover letter explaining the purpose of the interim 
evaluations. Some field offices may view this 
requirement as excessive due to their already heavy 
administrative workload. The time spent preparing the 
interim evaluations should be offset by time saved 
when improved contractor performance decreases the 
administrative burden. The interim evaluations are 
also useful and should be required when the AROICC on 
a contract is replaced. 

3 . Contractor Notification — NAVFAC should provide all 
contractors a copy of their performance evaluations. 
The feedback value of these evaluations should not be 
wasted. Knowing the evaluations are sent to the con- 
tractors should increase the AROICC' s objectivity when 
preparing them. 
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4. Training — Field offices should train AROICC's on how 
to complete evaluations. The AROICC's should also 
learn the purposes and standards for the evaluations. 
This training can be conducted at the field level 
using the policy statement as a guide. The training 
will improve the consistency, completeness, and relia- 
bility of the performance evaluation process. 

5. Management Emphasis — Management personnel should rate 
their AROICC's on how well they prepare and use the 
performance evaluations. This increased management 
emphasis on the evaluations will improve their use as 
a source of information about prospective contractor 
performance histories. AROICC's will take more care 
in preparing contractor evaluations if they know their 
personal performance rating is at stake. 

6. Reward System — All contractors rated outstanding 
should receive an award for their work. The awards 
should be meaningful and immediate. The contractors 
should know in advance what award is given for 
outstanding per-formance. In the short term, types of 
awards that can be given include: 

a. Special command recognition in the form of news- 
paper articles, special ribbon cuttings, letter 
of commendation, etc. 

b. Recommendation to other clients about the con- 
tractors outstanding work. 

7. Publicize the System — When performance evaluations are 
used to deny an unsatisfactory contractor a contract 
award NAVFAC should publicize it. Too many people 
neglect the evaluation system because they feel it 
doesn't work. 

8. Justify Satisfactory Ratings — A narrative should be 
required for satisfactory evaluations also. AROICC's 
should cite a contractor's strong and weak points even 
if the rating is satisfactory. That information is 
useful for predicting performance on future contracts. 
Not requiring a narrative on satisfactory ratings 
gives AROICC's pressed for time a quick means of com- 
pleting the evaluation requirement. 



C. LONG-TERM RECOMMENDATIONS 

1. Modify the SF 1420 — The current form does not evaluate 
contractors in enough specific areas. The form needs 
more rating categories to differentiate the marginal 
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from the satisfactory and highly satisfactory contrac- 
tors. A draft modification of the form is included as 
Appendix K. 

2 . Rewards — In the long term NAVFAC should be able to 
develop some tangible and meaningful reward for 
contractors who perform outstanding work. The con- 
tractor community could be consulted for suggestions. 

3. Improve the Data Base — Manual record keeping of con- 
tractor performance information is inadequate. Very 
few of the contract specialists surveyed use the EFD 
files of contractor performance evaluations. A com- 
puterized data base would be used (6 of 7 supervisory 
civil engineers said they would use it) . The data 
base should include: 

a. A summary of each contractor's evaluation ratings 

b. One line statements of contractor strengths and 
deficiencies . 

The data base should be accessible to field offices 
without significant assistance from EFD personnel. 
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APPENDIX A 



DEFINITIONS 



AROICC — Assistant Resident Officer-in-Charge of Construc- 
tion. Within the NAVFAC system this term refers to a Civil 
Engineer Corps Officer who directly administers construction 
contracts. Within the context of this study the term AROICC 
refers to both military and civilian personnel who directly 
administer construction contracts. 

Supervisory Civil Engineer — The senior civilian manager of a 
contracting field office. A senior Civil Engineer Corps 
officer is normally also assigned to manage in most NAVFAC 
field offices. 

Contract Specialist — Within the context of this study a 
contract specialist works in field offices: preparing con- 
tracts for advertisement and award, doing pre-award surveys, 
resolving contractual problems, and otherwise supporting the 
contracting efforts. 

OICC — Officer-in-Charge of Construction. Usually a senior 
Civil Engineer Corps Officer with delegated contracting 
authority. By virtue of that authority he is responsible 
for the success of all contracting actions he issues. The 
OICC usually prepares personnel performance evaluations on 
AROICC' s within his jurisdiction. The OICC also can recom- 
mend denying a contractor an award based on his determina- 
tion that the contractor is not responsible. 
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APPENDIX B 



STANDARD FORM 1420 



FOR OFFICIAL USE ONLY 

tWHEH COMPLETE!] I 



1. CONTRACT NUU8ER 

PERFORMANCE EVALUATION - CONSTRUCTION CONTRACTS 


part" - GENERAL CONTRACT DATA 


2. CONTRACTOR (Nom«, and ZIP coda) 
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OF ^ 

contract 

10*444) 




A. A0VERT1SE0 


I 


a.NEGOTIATEO 

FIRM 

D*" □ price nSES, 


4. COMPLEXITY 
1 ! DIFFICULT 


OF WORK 

| 1 ROUTINE 



3. DESCRIPTION AND LOCATION OF WORK 



6. FISCAL 
OATA 


► 


;a. amount of basic 

CONTRACT 

s 


8. total amount OF 

MODIFICATION 

$ 


C. LiOUIO A TED damages 
ASSESSED 

$ 


O. NET AMOUNT PAID 
CONTRACTOR 

$ 


7. SIGNIFICANT 
DATES 


H 


A. OATE OF AWARD 


8. ORIGINAL CONTRACT 

completion date 


C. REVISED CONTRACT 

completion DATE 


O. DATE WORK ACCEPTEO 



g. fv>E ano ext£nt"o> suecoNT r acting’ 



part H - PERFORMANCE EVALUATION OF CONTRACT tChtck topmonan box) 



9. PERFORMANCE ELEMENTS 


outstanding 


satisfactory 


unsatisfactory 


A. QUALITY OF work 








9. TIMELY PERFORMANCE 








C. EFFECTIVENESS OF MANAGEMENT 








o. compliance with maoR standards 








e. compliance with safety standards 









r6rOVERALL - EVALUATION . 
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11 EVALUATED 8Y 
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a. NAME AND TITLE <Typ« or prut t) 



A. ORGANIZATION (Typo or pnnt) 

\ 



a. NAME AND TITLE fTyp* or print) 




C. SIGNATURE 



12. EVALUATION REVIEWED BY 



c. signature" 



FOR OFFICIAL USE ONLY 

(WHEN COMPLETED I 



D. DATE 



O. DATE 



STANDARD FORM 1420 (10-33) 
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FAR (43 CFR) 33.236-1(0) 
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FOR OFFICIAL USE ONLY 

iWHtN COMPLETED) 



13. rtEMAHKb ON OU TSTANQING PERFORMANCE AS INOICATEOci* THfi CONTRA( rQO-«SP€R*-Q*MAINCe,‘N fh.SCONTMAtr F 'OIJ C n N- 
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APPENDIX C 



AROICC SURVEY 



1. Approximately how many satisfactory, outstanding, and 
unsatisfactory contractor performance evaluations have you 
written during your career as a contract administrator? 

Satisfactory Outstanding Unsatisfactory 

2 . Does your EFD have an instruction governing contractor 
performance evaluations? YES/NO 

If yes, how much help does it provide when you are 
filling out an evaluation? (CIRCLE ONE) 

A BIG HELP SOME HELP A LITTLE HELP NO HELP 

3. Listed below are some possible uses of performance 
evaluations. Please rank them in order from (1) meaning the 
most important use, to (5) meaning the least important use. 
(If you add a use in the other spot, rank from 1 to 6) 

Use to Provide Feedback to the Contractors 

Use to Support Findings of Non-Responsibility, and thus 

Deny Contractors an Award 

Used by Contract Specialists During the Pre-Award Survey 

Use to Give AROICC 's Information About a Prospective 

Contractor 

Use to Satisfy the FAR Requirement to Complete an 

Evaluation 

OTHER (Please Specify) 

4 . Have you ever received any training on how to complete 
the performance evaluation form? YES/NO 

If yes, which of these statements most accurately 
describes the level of your training? (CHECK ONE) 

Formal Classroom Training 

Briefed during ROICC Staff Meeting 

OJT by Contract Specialist or another AROICC 

Given a Copy of an old Evaluation as a Guide 

Other (Please Specify) 

5. Do you ever review a contractor's record of past per- 
formance prior to the start of a new contract? (YES/NO) 
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If yes, approximately how often do you do so? (CIRCLE 
ONE) 

Always 75% of 50% of 25% of 5% of 

the time the time the time the time 

6. Do you know where you could find a file of all of a 

particular contractor's past performance evaluations? 
YES/NO Where? 

7. On a scale of 1 to 10, how effective do you think the 
existing performance evaluation system is at improving the 
overall performance by NAVFAC contractors? (CIRCLE A 
NUMBER) 

123456789 10 

Not Effective Very Effective 

8. Have you ever written an interim evaluation on a 

contractor mid-way through a project? YES/NO 

If yes, what effect does the interim evaluation have on 
the contractor's performance? (CIRCLE ONE) 

Improvement Some Improvement Little Improvement No Change 

9. Do you think interim evaluations would help you to 
positively influence a contractor's behavior? YES/NO 

Why? 



10. Which of these statements most accurately describes how 
much emphasis your office management places on the perform- 
ance evaluations? (CHECK ONE) 

Evaluations Are Stressed Frequently 

Evaluations Are Stressed Occasionally 

Evaluations Are Stressed When A Contractor's Performance 

is UNSAT 

Evaluations Are Stressed When They're Overdue to the EFD 

Evaluations Are Never Stressed 

11. What comments do you have about the way we evaluate our 
contractors? Please include any suggestions you have for 
improving the evaluation process. (Use back of page if you 
need more room. ) 
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APPENDIX D 



CONTRACTOR SURVEY 



1. On a scale of 1 to 10, how much value do you assign to 
the performance evaluation rating you receive on a project? 
(CIRCLE ONE) 

123456789 10 
No Value High Value 

2. Of those you've received from NAVFAC , how objective and 
well documented were your performance evaluations? (CHECK 
ONE) 



Very Objective and Well Documented 

Pretty Objective and Well Documented 

Not very Objective or Well Documented 

Subjective. No Documentation 

3. Of the performance evaluations you've received from 
NAVFAC, did they provide you with useful feedback about your 
performance? YES/NO 

If YES, how was the feedback useful? 



4 . An interim evaluation is one provided mid-way through a 
project. How useful do you think interim evaluations would 
be in providing you feedback about your performance on a 
contract? (CHECK ONE) 

VERY USEFUL 

FAIRLY USEFUL 

USEFUL 

OF LITTLE USE 

NOT USEFUL 

5. Would you like to see NAVFAC issue interim performance 

evaluations on its contracts? YES/NO WHY? 
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6. Do you have a system for evaluating your subcontractors? 
YES/NO 

If yes, could you please describe your system or enclose 

some information about your system (i.e., rating forms)? 



7. Would you find it useful to receive a copy of your 
performance evaluation for each contract you perform? 
YES/NO 

Why? 



8. What comments do you have about the way NAVFAC evaluates 
its contractors? Please include any suggestions you have 
for improving the evaluation process. 
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APPENDIX E 



CONTRACT SPECIALIST SURVEY 



1. What % of the time do you refer to a contractor's past 
performance evaluations to assist you during the Pre-Award 
Survey process? 



0% 10% 30% 50% 70% 90% 100% 

On what types of awards (i.e., unknown contractor, MCON job, 
etc. ) 



2. What % of the time do you discuss the pre-award survey 
results with the AROICC assigned to the project? 

0% 10% 25% 50% 70% 90% 100% 



Under what circumstances do you do so? 

3. What source/sources do you use to find copies of a 
contractor's past performance evaluations? (i.e., EFD file, 
ROICC file) 

4. About what % of the time do you use your EFD's past 
performance evaluation files when doing a Pre-Award survey? 

0% 10% 25% 50% 75% 90% 100% 

Under what circumstances? (i.e., MCON job) 

5. How would you rate the quality of the performance evalu- 
ations you've seen in your office? 

Do you think they'd stand up in court in support of a non- 
responsibility determination? 

6. How much emphasis is placed on the evaluation process 
within your office? 

A LOT 

A FAIR AMOUNT 

SOME 

A LITTLE 

NONE 
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7. In the past year, how many contract awards have you 
withheld from the apparent low bidder because of his record 
of past performance? 

Was it easy to do? Y/N Why not? 

Would it have helped if you had 3 or 4 well-documented 
unsats to reference in your request for award denial? Y/N 

8 . What comments or suggestions for improving the evalua- 
tion process do you have? 



Can you give me the names of 

1. Sup Civil How long been on job 

2. Civilian Contract Administrator 

How long 
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APPENDIX F 



PACIFIC DIVISION CONSTRUCTION CONTRACTOR 
PERFORMANCE EVALUATION REPORT — SUPPLE- 
MENTAL INFORMATION SHEET 

PACNAVFACENj^OM^gST 4330.4 

CONSTRUCTION CONTRACTOR PERFORMANCE EVALUATION REPORT 
^SUPPLEMENTAL INFORMATION SHEET 

Code: 0 = Outstanding E = Excellent S = Satisfactory 

M = Marginally Satisfactory U = Unsatisfactory 



FACTOR 

1. Ability to prepare and timeliness in submission 

of schedule of prices, bill of materials and 
progress charts/networks 

2. Ability to submit shop drawings, brochures, 

and catalog cuts in accordance with contract 
requirements 

3. Ability to follow up and control equipment and 

material requirements 

4. Ability to deliver materials to jobsite on time. 

5. Quality of Management: a. Field Management . . 

b. Office Management . . 

6. Quality of Workmanship 

7. Compliance with security requirements 

8. Timely completion of contract 

9. Capability to provide adequate equipment and 

tools as needed 

10. Capability to regain lost time when behind 

schedule 

11. Capacity to secure adequate labor 

12. Compliance with safety standards 

13. Ability to pay subcontractors, material and 

personnel 

14. Timely response to Government requests . . . . 

15. Overall performance rating 

Remarks 
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APPENDIX G 



PACIFIC DIVISION EVALUATION RECORD FORM 



APPENDIX H 



SUPERVISORY CIVIL ENGINEER SURVEY 



1. How well does your EFD instruction governing contractor 
performance evaluations help the AROICC's when they're 
filling out the evaluations? 



2 . What changes would you suggest to improve the usefulness 
of the instruction? 



3. Do you provide your AROICC's any training on how to 
complete the performance evaluation form? YES/NO (Circle 
One) 



What kind of training? 



4. Does your office keep separate performance evaluation 
files on each contractor? YES/NO (Circle One) 

5. On a scale of 1 to 10, how effective do you think the 
existing performance evaluation, system is at improving the 
overall performance by NAVFAC contractors? (CIRCLE A 
NUMBER) 

1 2 3 4 5.6 7 8 9 10 

Not Effective Very Effective 

6. Does your office write interim evaluations on contrac- 
tors mid-way through a project? YES/NO 

If yes, what do the interim evaluations usually have on 
the contractor's performance? 



7 . Do you think NAVFAC should issue interim evaluations on 
all of its construction contracts? YES/NO 

Why? 
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8. What do you think of the form we use for contractor 
performance evaluations (SF 1420)? Would you make any 
changes to the form, and if so what changes? 



9. If you had access using the office computer to a file 
listing all of a contractor's past performance evaluations, 
would you use it to get a feel for how a prospective con- 
tractor of yours usually performs? YES/NO 

10. Do you think we should give our contractors a copy of 
their performance evaluations at the end of a contract? 
YES/NO Why or why not? 



11. What comments do you have about the way we evaluate our 
contractors? Please include any suggestions you have for 
improving the evaluation process. (Use back of page if you 
need more room) 
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APPENDIX I 



SCHMODE MEMORANDUM ON CONTRACTOR 
~ PERFORMANCE REPORT (CPR) 






OfNAV 5216/144A [Km. 8-81) 
S/N 0107 Lf -082 2320 



DEPARTMENT Of THE HAVy 

Memorandum 



OATt: 26 Nov 35 

«0M: Code R -1 

T0: Distribution List 



SUBJ: CONTRACTOR PERFORMANCE REPORT (CPR) 

Ref: (a) NORTHNAVFACENGCOMINST 4335.5 Subj: Performance Evaluations of 

Construction Contractors 

Enel: (1) Sample CPR Form 

1. We are tasked by reference (a) to evaluate the performance of our various 
construction contractors at the conclusion of their contract. The specific 
report required by reference (a) is fine for the purpose intended but does 
not provide use with the the information that would prove useful in 
fulfilling our administrative responsibilities for follow-on contracts with 
the same contractor. The five performance elements identified in the 
reference (a) required report are much too general and the wide dispersion 
between “satisfactory* and “unsatisfactory - leave much to be desired. 
Basically, we need to know who our marginal contractors -are, who our 
satisfactory contractors are that have some marginal traits and what these 
marginal or unsatisfactory traits are. If we know in advance of the specific 
areas in which our contractors have proven themselves to be less than 
satisfactory, we can give our attention to those specific areas including 
emphasis during the Preconstruction Conferences. In short, we need a system 
that can be accurate, objective, consistent, require a minimum of effort, and 
can be cumulative. 

2. Talking to a number of you, the following 12 factors seem to cover the 
majority of the significant elements that should be our concern: 

(1) Superintendent. 

(2) Safety. 

(3) Timely Completion. 

(4) Shop Drawings. 

(5) Completion of “Punch List*. 

(6) As-Built Drawings. 

(7) Change Order Negotiations. 

(8) Cooperation. 

(9) DRIs. 

(10) Payrolls. 

(11) Quality Assurance. 

(12) Technical Capability. 
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A brief description of what is intended for each element is in order. 

Superintendent . On site when work in progress; coordinates work of 
subcontractors; expedites work? controls project, etc. 

Safety . Site cleanliness; safety meetings; enforces hard-hats; accident 
reports, etc. 

Timely Completion . Project B.O.D.'d within current contract completion date. 

Shop Drawings . Timely submission and complete; contains KR certification 
stamp? submittal log. 

Completion of Punch List . Completion within reason after BOD; works on own 
initiative . 

As-Built Drawings . Keeping current as job progresses? timely submission at 
BOD. 

Change Order Negotiations . Timely response to RFPs; reasonableness of cost 
breakdown? willingness to negotiate a fair and reasonable price and time. 

Cooperation . Willingness to work with inspector and project engineer. 

DRIs . Timely submission and completeness. 

Payrolls . Timely submission without discrepancies. 

Quality Assurance . Workmanship and willing compliance with plans and specs. 

Technical Capability . Competency of contractor to do the work; ability to 
provide adequate workmen and equipment. 

3. A simple grading system that is self-explanatory is as follows: 

Outstanding - 5 
Above Average - 4 
Satisfactory - 3 
Marginal - 2 

Unsatisfactory - 1 

It is expected that a majority of our contractors will fall in the 
•Satisfactory* category. As previously stated, it is those contractors that 
have certain elements that fall in the "Unsatisfactory" or "Marginal" 
category that we are primarily concerned with. At the time we complete the 
more formal Construction Contractor Performance Evaluation Report, the 
project engineer and the project inspector will collectively complete this 
new Contractor Performance Report (CPR) by assigning numerical "grades" to 
each of the 12 elements. If a grade is either 1 or 2, a very brief 
explanation is to be provided. The combined effort of the inspector and the 
project engineer should not require much over 15 to 20 minutes. The CPR will 
be prepared for all construction contracts. The cumulative results of these 
reports will be posted in our computer and will be available for immediate 
recall for use on subsequent contracts with the contractor involved. 
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4. A sample of the CPR form to be used is attached as enclosure (1). The 
form provides for the indication of the general classification of the type of 
project involved, e.g., electrical, painting, general, etc. Please advise if 
you have any suggestions on how to improve this reporting system or the form 
iteself . This report is for internal use only arni^ the information will 
neither be forwarded to Northern Division nor /gyven /directly to the 
contractor involved. 




G. W. SCHMODE 
Supvry Civil Engineer 



DISTRIBUTION LIST: 
Project Engineers 
Inspection Staff 
R-20 

NORTHDIV (Code 05) 
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<0159P 



OICC/ROICC NEWPORT 



CONTRACTOR PERFORMANCE REPORT f C?R ) 



DATE: 



CONTRACTOR: BOD DATE: 



CONTRACT NO.; ACTIVITY: 



BRIE? TITLE: TYPE PROJECT: 



ELEMENT 


PERP. 

GRADE 


EXPLANATION IP 2 OR 1 


Superintendent 




Name : 


Saf etv 






Timely 

Completion 






Shop Drawinqs 






Punch List 
Completion 






As-Built 

Drawings 






Change Order 1 

Negotiations 






Cooperation 






DRIS 






Payrolls 






quality 

Assurance 




l 

| 


Technical 

Capability 






OVERALL: 







5 * Outstanding 
4 * Above Average 
3 * Satisfactory 

2 » Marginal 

1 - Onsatisfactory PROJECT INSPECTOR 



PROJECT ENGINEER 

Pile Copy 

R-l RO-1 (Data Entered) 
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59P 



QICC/ROICC NEWPORT 



5AWL& 



CONTRACTOR PERFORMANCE REPORT ( C7R ) 



DATS: 3 f 'J'fn fC 

CONTRACTOR : / ff J f 6 *i C.3 aziL-Jla. BOD DATE: ^ 



CONTRACT NO.; /_7f^V_ ACTIVITY: /Wr.-V- 



BRIEF TITLE: /W J / 1? t/ <1 7 aJ C tf/firr /* er n(7 TYPE PROJECT: a b 


1/ j 


ELEMENT 


PERP. 

GRADE 


EXPLANATION IF 2 OR 1 


Superintendent 


7 


» cf/t olrcr*./ ffie ! 


Safety 


2- 


( * / / / 


Timely 

Completion 


3 


- — '77 |! 


Shop Drawings 


? i 


i 

i 


Punch List 
Completion 


2 1 


/(ft £ X2C r pT/o* a f/y s/olaJ iri f K0±) S’ ficT 7 */ f!h /7*M/ r « 


As-Built 

Drawings 


3 1 


/ / / / _ 7 7 


Change Order I 

Negotiations 


3 




Cooperation 


3 




ORIs 


2 


/ Q yT //f 3 D&J r 7n V^//C hoTtS 


Payrolls 


3 1 


J ' j: 


Uuality 

Assurance 






Technical • 
Capability 






OVERALL: 




otiose, f f/of / 77) c /f/< rct/c*32*X /Tic | 

7h+T A A S-t. jaS" r-riAw;U#(< Cc~ csr/suffL, atcTi/c-ctZs, fraT A/** (tXc j 


5 - Outstanding 
4 ■ Above Average 

3 ■ Satisfactory sn s J 

2 - Marginal /^A /r/ 


1 - Ons 

File Qfy 

R-l Xl RO-1 (Data Ei 


atisfactory ' PROJECT^ SP ECTOR 


V - PROJECT ENGINEER 

ntered) V K\ 
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APPENDIX J 



DRAFT POLICY STATEMENT 



A. INTENT: The Federal Acquisition Regulations and the 

Department of Defense FAR Supplement direct NAVFAC to issue 
contractor performance evaluations on most of its 

construction contracts. This policy statement is intended 
to: 

(1) Establish the purposes for which NAVFAC intends to 
use the contractor performance evaluations. 

(2) Provide standards of performance necessary to achieve 
the various performance ratings. 

(3) Emphasize the activities included in each evaluation 
item. 

(4) Promulgate direction for distribution of contractor 
performance evaluations. 

This statement should increase the awareness, under- 
standing, reliability, objectivity, and usefulness of the 
contractor performance evaluations. Comments about this 
policy statement should be addressed to: 

Code 022 

Naval Facilities Engineering Command 
200 Stovall Street 
Alexandria, Va 22332 

B. USES: The contractor performance evaluations will be 

used: 

(1) To support determinations of contractor responsi- 
bility during pre-award surveys. 

(2) To provide contractors feedback about their ongoing 
performance . 
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(3) By AROICC's to assess a contractor's strengths and 
weaknesses prior to administering a new contract. 

C. STANDARDS: NAVFAC will augment the three existing 

rating categories on the Standard Form 1420 by adding 

categories of marginal and highly satisfactory . These 

standards describe typical performances worthy of each 

rating: 

Outstanding — The contractor should meet or exceed all 
contract requirements; display a high level of 
professionalism in all aspects of performance; 
anticipate all potential problems within the 
realm of his responsibility and implement 
effective solutions; cooperate fully (within 
the requirements of the contract) with the 
AROICC; minimize the amount of government 
administrative effort on the contract. 

Highly 

Satisfactory — The contractor should: meet or exceed all 
contract requirements; display professionalism 
in all aspects of performance; anticipate most 
potential problems within the realm of his 

responsibility and implement effective solu- 
tions most of the time; cooperate (within the 
requirements of the contract) with the AROICC; 
minimize the amount of government administra- 
tive effort on the contract. 

Satisfactory — The contractor should: meet all contract 

requirements; display an average amount of 
professionalism; anticipate some problems 
within the realm of responsibility and imple- 
ment effective solutions some of the time; 

cooperate (within the requirements of the 
contract) most of the time; require only a 
normal amount of government contract adminis- 
trative effort on the contract. 

Marginally 

Satisfactory--The contractor should: meet most of the 

contract requirements, including those criti- 
cal to the use of the particular facility; 
display some professionalism; cooperate 
(within the requirements of the contract) 
minimally with the AROICC; react to most 
problems rather than anticipate them; require 
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an excessive amount of government administra- 
tive effort on the contract. 

Unsatisfactory — The contractor has failed to meet the mini- 
mum contract requirements even after signifi- 
cant administrative effort by the government. 
The contractor was uncooperative, failed to 
anticipate problems within the realm of his 
responsibility, and did not attempt to develop 
solutions. The contractors actions were not 
professional . 

D. EVALUATION ELEMENTS — Within the 5 broad evaluation 
elements on the SF 1420 contractors will be specifically 
rated on: 



Broad Element 



A. Quality of Work 



B. Timely Performance 



C. Effectiveness of 
Management 



D. Compliance With 
Labor Standards 

E. Compliance With 
Safety Standards 

Although these specific 
evaluated using the SF 142 



Specific Element 



1. Materials Used 

2 . Workmanship 

3. Quality Assurance Plan and 

Efforts 

1. Scheduling 

2. Punch List Completion 

1. Effectiveness of 

Superintendent 

2. Cooperation 

3. Administration (i.e., sub- 

mittals, payrolls) 

1. Payrolls 

2. Violations 

1. Safety Awareness and 

Attitude 

2. Safety Plan 

3. Safety Record 



items may have been implicitly 
, they will now be explicitly 



evaluated using this modified system. The contractors and 
AROICC's both know what areas of performance will be 
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evaluated. The general standards can be applied to each 

specific evaluation element. As an example, the following 

standards could apply to Quality of Work-Materials Used: 

Outstanding — All materials met or exceeded specific quality 
requirements. Complete and acceptable 

material submittals were made as required. 
Manufacturers recommendations and appropriate 
industry practices were followed when using 
all materials. Obvious care was taken when 
storing and handling materials. No government 
effort was needed to ensure compliance with 
materials requirements. 

Highly 

Satisfactory — All materials met or exceeded specified qual- 
ity requirements. Complete and acceptable 
material submittals were made as required. 
Care was taken when storing and handling 
materials. Manufacturers recommendations and 
appropriate industry practices were followed 
most of the time. Very little government 
effort was needed to ensure compliance with 
material requirements. 

Satisfactory — All materials met or exceeded specified qual- 
ity requirements. Complete and acceptable 
submittals were made as required. Some resub- 
mittals were needed before final material 
approval was granted. Normal care was usually 
taken when storing and handling materials. 
Manufacturers recommendations and industry 
practices were usually followed. Government 
effort was required occasionally to ensure 
compliance with material requirements. 

Marginally 

Satisfactory — All materials met specified quality require- 
ments. Complete and acceptable submittals 
were made only after several attempts. Below 
average care was taken when storing and 
handling materials. Manufacturers recommen- 
dations and appropriate industry practices 
were occasionally violated. A large amount of 
government effort and some rework was required 
to ensure compliance with material 
requirements . 

Unsatisfactory — Not all materials met specified quality 
requirements. Material submittals were 
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incomplete and often unacceptable. Little 
care was taken when storing and handling 
materials. Manufacturers recommendations and 
appropriate industry practices were not 
routinely followed. A significant amount of 
government effort and rework was required to 
bring the contractor into compliance with 
material requirements. 

E. PREPARATION AND DISTRIBUTION: When reasonable to do so, 
or when requested by a contractor, the AROICC will issue an 
interim evaluation at the mid-point of the contract 
duration. The interim evaluation is intended to provide 
contractors feedback about their performance. The contrac- 
tors final performance evaluation rating is intended to rate 
the contractors performance over the entire period and 
therefore should not depend necessarily on the interim 
evaluation ratings. 

Interim evaluations should be sent to the contractors at 
the mid-point of the contract duration. If a contract is 
for an extended period of time, interim evaluations should 
be written at least for every 6 months of performance. 
Final performance evaluations will be sent to contractors at 
the time of final payment. 
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APPENDIX K 



DRAFT REVISION TO STANDARD FORM 1420 



These modifications are recommended for improving the 
Standard Form 1420. 

1. Items 1-8, 11 and 12 should remain unchanged, but 

the vertical space allowed for items 5, 8, 11, and 12 should 
be reduced to accommodate the changes to items 9 and 10. 

2. Change item 9 to look like this: 

9. Perform- Out- Highly Satis- Margi- 

ance stand- Satisf. factory nally 

Elements Satisf. 

A. Quality of Work 

1. Materials 

2 . Workmanship 

3. Q/A plan 

B. Timely Performance 

1. Scheduling 

2 . Punch List 

C. Effectiveness of Mgmt. 

1 . Superintendence 

2 . Cooperation 

3. Administration 

D. Compliance w/Labor 

1. Payrolls 

2. Violations 

E. Compliance w/Safety 

1. Awareness/Attitude 

2. Safety plan 

3. Safety record 

3. Combine items 13 and 14 into one explanation and 
justification section. Require narrative supporting all 
ratings, not just outstandings or unsatisfactories . 



Unsatis- 

factory 
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